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FOREWORD 

In June 2002, ENQA organised a workshop about Benchmarking in Higher Education 
in Finland, and produced the second Workshop Report entitled Benchmarking in the 
Improvement of Higher Education. Almost ten years later and in the same country, ENQA 
held its fifth annual Internal Quality Assurance Seminar on benchmarking in internal 
quality assurance of agencies. This demonstrates that benchmarking is still considered 
as an interesting subject to promote. 

In an increasingly competitive higher education sector, benchmarking is a modern 
management tool to support strategic decision-making, yet its use is still too limited. 

In June 2011, the annual seminar of the ENQA IQA Group thus focused on the theme of 
Learning from each other - using benchmarking to develop IQA. 

Agencies involved in the IQA Group were invited, in small groups, to compare 
practices and expertise in their agencies as well as common challenges encountered in 
their daily operations. 

The report presents a general overview of the benchmarking theme and discusses 
common features and differences of the benchmarked agencies' IQA activities in terms 
of the selected three themes: performance indicators, follow-up on feedback and staff 
competence/development. The report also puts forward the benchmarking partners' 
views on strengths, weaknesses and recommendations for development of each other's 
activities, as well as the good practice they have identified on the selected theme. 

ENQA fully supports and encourages the open sharing of good practice and challenges 
among its membership in order to improve both internal and external processes. 

I hope that this publication, illustrating how collaborative benchmarking can be applied 
in higher education and can benefit quality assurance agencies, will be of interest and use 
to agencies in contributing to the enhancement of their internal quality assurance. 


Achim Hopbach 

President 

European Association for Quality Assurance in Higher Education (ENQA) 
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INTRODUCTION 

The Internal Quality Assurance group of ENQA (IQA Group) has been organising a 
yearly seminar for its members since 2007. Staff members involved in IQA of all ENQA 
members can join the activities of the Group. The main objective is to share experiences 
concerning the internal quality assurance of work processes in the participating agencies. 
The Group is coordinated by a Steering group (SG), consisting of five members. The 
composition of the Steering group changes gradually by election of one or two members 
every year. 

The overarching theme of the 2011 seminar was how to use benchmarking as a tool for 
developing an agency's internal quality assurance system. The seminar gathered around 
45 participants in the premises of the Finnish Higher Education Evaluation Council 
(FINHEEC) in Helsinki on 16-17 June 2011. 

"Benchmarking involves comparing different aspects of the work of a group of 
organisations. It can be a very flexible approach. You can compare services, products or 
processes; you can look at a wide range of issues or focus on areas of concern; and you 
can benchmark with similar organisations or take a cross-sector approach on common 
issues such as customer care. Benchmarking may take place as a one-off exercise or 
be an ongoing relationship. The benchmarking exercise should be a mutually beneficial 
relationship, with every organisation in the benchmarking group being able to learn and 
develop from the experience of others." 1 

The Steering group based the preparation of the benchmarking activity on this 
definition. Agencies which are similar to each other, i.e. in size or scope, were grouped 
in pairs or triplets. Each group included an agency member of the Steering group. They 
compared their own practice with others on a certain focus area before the seminar, 
between January and May 2011. In addition to good practices, the participating agencies 
were encouraged to openly share which processes they find challenging or ineffective in 
their agencies. The findings were presented in the IQA Seminar in June 2011. 

The benchmarking exercise focused on the following areas: 

• Benchmarking of performance indicators (with FINHEEC) 

• Benchmarking of on the follow-up of feedback (with ACSUCYL) 

• Benchmarking of staff competence/development (with NVAO) 

The present report gathers good practise and expertise related to these three themes: 
follow up on feedback (chapter 2), staff development (chapter 3) and performance 
indicators (chapter 4). The first chapter of the report provides a general overview of the 
benchmarking theme and is based on the keynote speech given by Dr Nadine Burquel. 


1 Source: The Institute of Chartered Secretaries & Administrators (ICSA), Charity trustees guide 
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CHAPTER 1: 

BENCHMARKING IN EUROPEAN 
HIGHER EDUCATION 

Nadine Burquel, European Centre for the Strategic Management of Universities (ESMU) 

INTRODUCTION 

Collaborative benchmarking is a valuable modern management tool for organisations 
eager to steer their institutional developments in a strategic way. It involves a process 
of target setting by the institutions themselves looking to increase their performance 
through inter-organisational learning. 

This article focuses on benchmarking in higher education. It starts with an outline of 
the origins of benchmarking and provides a brief review of the literature. It explains how 
benchmarking can be applied in higher education. A number of suggestions are then 
made on how quality assurance agencies could benefit from the practice of collaborative 
benchmarking. The article is based on the findings of a EU-funded project (DG Education 
and Culture) Benchmarking in European Higher Education carried out from 2006 to 
2011 (Phases I and II) by a consortium led by the European Centre for the Strategic 
Management of Universities (ESMU) 2 . 

THE ORIGINS OF BENCHMARKING IN HIGHER EDUCATION 

Benchmarking originated in the private sector in 1979. In a context of severe financial 
difficulties. Xerox Corporation started using benchmarking to try to understand why 
competitors were performing better. This process led to major changes to improve 
internal processes and enabled the company to regain a strong market position. Since 
then, benchmarking has been widely used in industry, manufacturing, finance, transport, 
logistics, retail and services. 

In the public sector, with the development of new public management, benchmarking 
has been increasingly used in the health sector, the public transport sector, in local and 
regional administrations and the like. At the European level, mechanisms have been 
developed to benchmark labour market policies, Europe's industrial competitiveness or 
public transport systems. 

Major changes have taken place in European higher education, resulting in higher 
education institutions having to enhance their attractiveness on the market and profile 
themselves much more strategically. Quality is a key to support these developments, 
and in this context, enhancing university performance through strategic management 
becomes crucial. However, systematic data collection on institutional performance to 
inform decision-making is still lacking in many higher education institutions. 

The purpose of quality assurance is to ensure accountability, yet it must also enhance 
the quality of higher education itself. The standards and guidelines for quality assurance 
in the European higher education area (ESG) provide directions for higher education 
institutions to improve their internal quality assurance policies and procedures, yet there 
is often a perception that European quality assurance has become too bureaucratised, 
failing to lead to real, deep changes in the sector. Not all higher education institutions 
take sufficient ownership in the process. 


2 www.esmu.be 
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Some implicit forms of benchmarking have always been part of higher education 
with various forms of peer review and site visits. What is new is the use of explicit 
benchmarking and the formalisation of processes. The growth of benchmarking in higher 
education reflects the search for continuous quality improvement and more effective 
ways of improving performance in an increasingly diversified higher education sector. 

The concrete nature of benchmarking as a self-improvement tool to improve 
organisational performance is not always fully understood. Benchmarking is often 
performed as a mere data gathering exercise lacking a systematic approach and target 
setting for institutional improvement. 

Benchmarking relates to other transparency tools such as classifications and rankings. 
With a focus on descriptive indicators, classifications make it possible for higher 
education institutions to identify adequate institutional partners with similar profiles for 
benchmarking exercises, thus leading to more relevant comparison between institutions. 
Reliable rankings can be at the starting point of benchmarking exercises for those 
institutions willing to increase their performance in the rankings. 

REVIEW OF THE LITERATURE 

In the first phase of our EU-funded project Benchmarking in European Higher Education 3 , 
we compiled 150 articles and references on benchmarking. Our first practical guide 
Benchmarking in European Higher Education (2008) produced a review of this literature. 

The enormous literature on benchmarking in higher education focuses mainly on the 
practice of benchmarking. The term is used for different practices from the comparison 
of statistical data to the detailed analysis of processes within institutions. Publications 
focus either on the character of benchmarking (i.e. an exercise focusing on institutional 
processes or on performance) or on the aim of the benchmarking exercise (to learn to 
improve internal processes or to improve one's competitive position). 

Stressing the wide range of diversity between higher education institutions, Yorke 
(1999:91) claims that there "can be no single reference point for the purposes of 
benchmarking". 

Alstete (1995) defines four types of benchmarking linked to the voluntary 
participation of institutions, i.e. international benchmarking, external competitive 
benchmarking, external collaborative benchmarking and external trans-industry (best-in- 
class) benchmarking. 

In Benchmarking in Higher Education, An international review, Schofield (1998) points to 
the difficulties with the definitions by highlighting that "the term can vary considerably 
between different approaches and practitioners, causing problems to institutions 
investigating the subject for the first time". 

UNESCO-CEPES (2007) uses similar descriptions referring to internal benchmarking, 
external competitive benchmarking, functional benchmarking (comparing institutional 
processes), trans-institutional benchmarking (across multiple institutions), implicit 
benchmarking (quasi-benchmarking looking at the production and publication of data/ 
performance indicators), generic benchmarking (looking at basic practice process or 
services) and process-based benchmarking (looking at processes by which results are 
achieved). 

In its report Benchmarking in the Improvement of Higher education (Hamalainen Kauko 
et al., 2002), ENQA attempts an understanding of the principles of true benchmarking. 

A long list of 32 attributes is given to benchmarking, the main ones being collaborative/ 
competitive, qualitative/quantitative, internal/external, outcome-oriented or experience- 

3 www.education-benchmarking.eu 
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seeking, with various purposes (standards, benchmarks, best practices) and interests (to 
compare, improve, cooperate), depending on the owners of the benchmarking exercises. 
The report concludes that "good instruments are needed for useful benchmarking 
exercises" and that "current benchmarking methodologies in Europe must be improved". 

THE DEVELOPMENT OF COLLABORATIVE BENCHMARKING 

We identified two types of benchmarking approaches in higher education. In the first 
non-collaborative type, higher education institutions call on consulting firms to buy data 
to compare their performance with other institutions. In the second type, benchmarking 
is carried out in a collaborative way as an inter-organisational learning process between 
institutions with a view to improving their modes of operation. This second approach 
requires a high level of trust and confidentiality between participating institutions. 

In Europe, collaborative benchmarking approaches in the higher education sector have 
developed from the mid-nineties as initiatives launched at the national level by groups of 
institutions or by independent bodies. These have usually only involved a small number 
of institutions. Transnational level exercises have so far remained limited. 

In our first benchmarking project, we analysed 18 collaborative benchmarking groups 
worldwide in Europe, Australia, and the United States. 

We identified the following 13 criteria: 

• institutional nature; 

• group character (homogenous or heterogeneous); 

• management (self-steered group or group managed with the support of an external 
moderating organisation); 

• group size; 

• group membership; 

• membership fees; 

• performance-based nature (or not); 

• timeline; 

• geographical scope; 

• methodology (quantitative or qualitative exercise); 

• focus (input, output, process); 

• level of participation; 

• dissemination of outcomes. 

We did not manage to identify specific models to characterise these benchmarking 
groups. Benchmarking groups all vary by aims, objectives, structure and methodology. 
Many groups struggle to find the right facilitator and lack appropriate human, technical, 
and financial resources. Even the most successful initiatives do not sufficiently make use 
of the results for decision-making purposes back in participating institutions. 

Building on this analysis, we defined benchmarking as the process of self-evaluation 
and self-improvement through the systematic and collaborative comparison of practice and 
performance with similar organisations in order to identify strengths and weaknesses, to 
learn to adapt and to set new targets to improve performance. 

This approach was adopted at the beginning of the second phase of our EU-funded 
benchmarking project which involved 41 European universities, divided in four groups on 
university governance, lifelong learning, curriculum reforms and university-enterprise 
cooperation. The outcome was a handbook titled Benchmarking in European higher 
education. 
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Benchmarking requires senior leadership commitment and willingness to improve 
institutional performance, a clear understanding of processes, reliance on a strong 
peer group and the commitment of financial and human resources to support the 
implementation of the exercise. Benchmarking involves six stages: 

• Strategic decision-making; 

• choosing appropriate partners; 

• defining priorities/focus and indicators; 

• data gathering and reporting; 

• developing action plans; 

• monitoring results. 

The most effective benchmarking exercises are with partner institutions of a similar 
profile sharing a common interest and a similar degree of development in a given area. 

It is crucial that the benchmarking group agrees on common priorities based on 
which a list of performance indicators can be developed. Depending on the nature of 
the benchmarking exercise, there will be a stronger focus on qualitative or quantitative 
indicators, or on input, process, output or outcome indicators. A full benchmarking cycle 
requires all types. The sets of indicators get final agreement from the senior leadership of 
each participating institution. 

There is also an agreement on what constitutes good performance with four 
"expertise levels": basic performance, standard performance, good and excellent 
performance. 

Once the priorities and indicators have been defined, the data gathering can start. 

The question is how much data should be gathered and how to ensure their validity and 
reliability. The group may wish to use external experts for this purpose. Once the data 
have been gathered, institutions are placed and scored against the "expertise levels". 

The comparative scorecards combining the individual university scores show current 
performance and provide indications on where individual institutions should place the 
precise focus for their strategic improvement plan. From a collective exercise the process 
becomes very individual with institutions drawing their own realistic action plans to 
address the gaps identified around a pilot project with a precise timeframe, specific 
tasks, and adequate financial and human resources. 

HOW CAN QUALITY ASSURANCE (QA) AGENCIES BENEFIT FROM 
BENCHMARKING? 

Quality assurance agencies can gain significantly from comparative benchmarking 
exercises to assess the effectiveness of their activities in responding to the improvement 
of quality in higher education at the national and European levels. 

Such benchmarking exercises between QA agencies would require the preliminary 
identification of specific priority areas and the choice of relevant partner agencies (i.e. 
with similar interests, activities and areas of institutional development). 

The data gathering, analysis and reporting would lead to comparative overviews of the 
agencies' performance, based on which action plans with targets for improvement can be 
defined. 

Benchmarking is a structured and collaborative learning experience which would 
help QA agencies identify and disseminate good practices and develop new ways of 
addressing specific problems. Such inter-organisational learning between QA agencies 
within the context of ENQA would enhance their reputation in demonstrating a 
continuous effort to improve the way in which quality assurance is performed in Europe. 
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Benchmarking exercises could also be used to create databases of good practices to 
support the implementation of the European Standards and Guidelines, to review QA 
agencies' procedures from the point of view of their adequacy to the quality agenda set in 
the context of the Bologna Process. It would help support newly established QA agencies 
with their work. 

CONCLUDING REMARKS 

In an increasingly competitive higher education sector, benchmarking is a modern 
management tool to support strategic decision-making, yet its use is still too limited. 

Whether carried out within or between institutions, benchmarking must always 
lie in the identification of strengths and weaknesses with a view to set targets for 
improvement. Benchmarking goes beyond the comparison of statistical data. It is a 
dynamic comparative exercise during which relevant indicators are defined, and against 
which the performance of a group of institutions can be measured. 

Benchmarking must be implemented at the strategic level to support strategic 
developments. It will only produce valuable results if placed in the context of 
organisational transformation and progress. The key is to define where efforts should be 
placed to maximise results and constantly set new targets for institutional improvement. 
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CHAPTER 2: 

LEARNING FROM EACH OTHER 
- USING BENCHMARKING TO 
DEVELOP IQA. FOLLOW UP ON 
FEEDBACK 

Sandra Marcos, Quality Assurance Agency for the University System in Castilla y Leon 
(ACSUCYL) 


INTRODUCTION 

With a view to sharing the information and pinpointing the good practices which Quality 
Agencies for Higher Education carry out concerning the processing and follow-up of 
information gathered from stakeholders, the possibility was considered of conducting a 
joint benchmarking exercise involving any member agencies of the IQA group who might 
be interested. 

Three agencies expressed an interest in taking part: AEQES (I'Agence pour I'Evaluation 
de la Qualite de I'Enseignement Superieur de la Communaute francaise de Belgique), 

VLIR (Vlaamse Interuniversitaire Raad) and OAQ (Swiss Center of Accreditation and 
Quality Assurance in Higher Education). 

These three agencies, together with ACSUCYL, a member of the IQA Steering Group 
and project coordinator, worked on the project for three months, which was concluded 
with the presentation of the results to emerge subsequent to an analysis of the relevant 
data, at the annual seminar organised by the IQA Steering Group. 

The goal of the exercise was firstly to reflect on whether the information we 
gather both internally as well as externally actually proves useful and valuable for 
the organisation and whether the tools used to gather said information are the most 
appropriate. Secondly, the goal of the exercise was to ascertain whether this information 
actually helps to improve the quality of the activities that the agencies carry out and, if 
so, to what extent and what tools are used to verify this. 

With these goals in mind, the four participating agencies got down to work. The initial 
stage involved exchanging information concerning how each agency worked as well as 
various aspects related to the issue the activity was concerned with. As a starting point 
for exchanging information, agencies responded to the following questions: 

• What methods do you use to collect information to get feedback? How do you use 
that information? 

• From whom do you collect feedback? 

• What are the strengths and weaknesses of these methods? 

• How are you thinking of developing these methods? 

• Do you feel the information is really useful for improving the quality in your agency? 
Do you make a real follow-up on how the information has helped your organisation 
to improve? If so, how? 

Once the various agencies had responded to the above questions, an analysis was carried 
out, first individually and later jointly, in an effort to pinpoint similarities and differences 
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which, in certain instances, were due to the areas each agency was involved in as well as 
the scope, mission and structure of each. 

In all, three meetings were held where the information provided by each agency 
and how they conducted their business were discussed. To conclude the exercise, and 
with a view to preparing the presentation to be given at the IQA seminar, the following 
questions were addressed: 

• What are the similarities and differences between the agencies involved in 
benchmarking? 

• What new ideas did you learn from this exercise? 

After the presentation at the seminar, several working groups were set up in order to give 
the participants a chance to express their opinion concerning the issue at hand, as well as 
to exchange good practices and share their concerns. 

Most of the information contained in the present article has been taken from the 
conclusions reached by the working groups comprising the four agencies and the working 
groups at the seminar. 

DEVELOPMENT 

The key to obtaining useful, accurate and valuable information lies in the agency's ability 
to have a clear idea of what kind of information is required, how it may best be obtained, 
who can provide it, and of course what is the final goal being pursued. 

It should be highlighted that most quality agencies for higher education use satisfaction 
surveys as a means of collecting information. These are sent periodically to stakeholders 
involved in the activity in question, and the feedback obtained helps in decision-making 
concerning the processes conducted and in general regarding the services provided. 

Yet, despite being the most commonly used approach, it is by no means the most 
efficient nor the most highly regarded. The fact that surveys are conducted anonymously 
is usually felt to be a positive aspect. However, unlike other methods, the limited nature of 
the questions does not allow for possible interaction with respondents nor does it lead to 
any discussion from which useful ideas may emerge. 

Some agencies organise focus groups. This method seems to be more efficient as 
the information gathered is more accurate, reliable and less biased. Some agencies 
even employ both methods, the responses obtained through the satisfaction surveys 
subsequently being analysed by focus groups. 

Another commonly used tool is meetings, both at formal and informal level. In 
this respect, the general feeling is that internal communication within the agencies 
themselves needs to be strengthened. In many instances, internal feedback is deemed 
more valuable than external feedback. 

Meta-evaluation was highlighted as one useful means of analysing information about 
processes. This involves appraising all the information related to an evaluation process 
with a view to pinpointing improvements to be implemented in the future. 

Other means of gathering information included seminars, congresses, etc. 

The kind of question posed in an effort to gather information was seen to vary greatly, 
although participants all agreed that one of the most relevant, and at the same time 
probably one of the most complex question concerns the impact of evaluations on the 
education system. 

The analysis conducted by the working group revealed that satisfaction surveys were 
used for stakeholders most closely involved in the agency's activities. However, for those 
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who were not so closely involved, other tools such as informal meetings, seminars, and 
discussion groups were used to gather information. 

One problem all participants face emerges when they need to identify from whom 
the information should be collected. It is clear that evaluators involved in the evaluation 
processes, as well as those requesting assessment, whether they are universities or 
private individuals seeking evaluation in addition to students, need to be surveyed, 
although not all agencies do indeed survey students. There is, however, a general feeling 
that not all of those whose opinion should be sought are actually surveyed. In most cases, 
this is due to the difficulty involved in pinpointing who should be surveyed amongst the 
particular groups such as society at large, or workers. However, difficulties may arise 
simply because of the problems related to interviewing certain authorities who, strangely 
enough, are in many cases responsible for taking the key decisions. 

This ties in with one of the challenges highlighted by the working groups concerning 
agencies' independence when undertaking changes. One of the issues that emerged 
during the presentation was the difference between the various agencies when it 
came to dealing with the changes to arise as a result of feedback. Various levels of 
independence were identified with regard to making changes, depending on whether 
these are internal or external. The analysis revealed that in the case of changes related to 
internal functioning, agencies normally have complete freedom to make such changes. 
Yet, in many cases, even when it is clear that certain changes are required to improve 
the processes, the former cannot be implemented because agencies lack authority to do 
so. This is usually the case when processes are imposed by legislation or by government 
authorities. 

In the working group sessions held during the seminar, this gave rise to the question 
whether it is worth gathering information on matters the agencies know well beforehand 
they cannot be changed. However, the fact that no direct change can be made does 
not mean that proposals should not be put forward. The competent authorities should 
be made aware of the opportunities for improving the processes. The agencies, in 
most cases, have the information on hand to make an informed judgement on whether 
processes and criteria need to be amended with a view to improving them. 

Subsequent to the analysis carried out, one of the challenges found by the working 
group to face all agencies was the lack of communication with regard to the outcomes to 
emerge both about the actual processes as well as the satisfaction amongst stakeholders 
on the agency's work. Society needs to be informed of the outcomes, and how this can 
be achieved is one of the challenges agencies must deal with. Certain agencies such as 
ACSUCYL do publish the outcomes obtained although there is no subsequent feedback 
in relation thereto. With this goal in mind, AEQES is organising discussion groups. 

RESULTS 

As a result of this exercise, the new ideas that emerged related to two areas: 

1. All agencies face similar challenges with regard to publicising outcomes, namely: 

- How to address society at large 

- How to determine who to talk to 

- Transparency 

2. Hearings and discussions with stakeholders about the outcomes of the surveys 
need to be held. As pointed out previously, it is interesting to obtain subsequent 
feedback on the outcomes. Some agencies like VLIR and AEQES have already 
started to do this through debriefing sessions. In order to lend greater weight to 
this issue, agencies should also include this question in their communication plan. 
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CONCLUSION 

Gathering information from all stakeholders concerning the activities agencies are 
involved in is something we all need to consider. Doing this correctly, with a clear focus 
on specific objectives, which should concur with those of the agency itself, using the 
appropriate methods and conducting proper follow-up of the emerging outcomes will no 
doubt have a beneficial effect and improve the activities we engage in. 

We should not overlook the problems agencies are facing when undertaking a project 
of this nature, such as deciding which stakeholders to consult, or how to obtain reliable 
information or higher response rates. 

Furthermore, even when agencies do obtain useful and reliable information which 
enables them to suggest changes aimed at improving the processes, they lack the 
independence required to undertake such changes. 

This should not, however, prevent agencies from continuing to work towards gathering 
information to improve, become more efficient and enhance the quality of their services 
as well as contributing towards the ongoing improvement of higher education. 
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CHAPTER 3: 

THE BENCHMARK OF STAFF 
DEVELOPMENT; A VALUABLE FIRST 
STEP 

Douglas Blackstock, Quality Assurance Agency for Higher Education (QAA) 

Marion Moser, Accreditation, Certification and Quality Assurance Institute (ACQUIN) 
Henri Ponds, Accreditation Organisation of the Netherlands and Flanders (NVAO) 


INTRODUCTION 

The central theme of the 2011 seminar was benchmarking. Current practices and 
expertise in the agencies were compared and discussed related to three themes: Follow 
up on feedback, Performance indicators and Staff development. This contribution will 
discuss the benchmark on Staff development. 

Five agencies participated in the benchmark on staff development: QAA from the 
United Kingdom, EVA form Denmark, ANECA from Spain, ACQUIN from Germany and 
NVAO from the Netherlands and Flanders. These agencies cooperated in advance of the 
seminar and presented a common result during the seminar. Hereafter the process and 
the main results of this benchmark on staff development are presented. 

PROCESS 

The preparation of the seminar and the benchmark activity was coordinated by the 
SG, which formulated five basic questions to be considered for all three, mentioned 
benchmark themes. The five questions, applied to staff development, are: 

• What core competences do you wish staff to achieve through staff development in 
your organisation? 

• What sort of staff development activities do you have and how are they organised? 

• What and with what methods do you collect information on Staff development ? 
How do you report and act on this information? 

• What are the strengths and weaknesses of your current practice? 

• How are you thinking of developing your practice? 

First of all, the five agencies involved in the staff development benchmark exercise were 
asked to answer these questions in a written contribution. Due to costs and the distance 
between the five agencies a practical way of deliberation was chosen. Two agencies, 

EVA and NVAO, were prepared to take care of progress and to involve the other three 
agencies in conducting the activity. Communication took place by e-mail and telephone. 

The separate contributions were analysed by EVA and NVAO and, as a result, 10 
concrete benchmark topics were indicated and proposed to the other three agencies. 
These benchmark topics guided the comparison that was carried out afterwards. 

The five agencies agreed on the following 10 topics: 

• Required competencies for staff; 

• Current competencies of staff; 

• Assessment of current staff; 

• Selection and guidance of new staff; 
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• Training and development on the long run; 

• Training and coaching on the job; 

• Rewards/incentives; 

• Human Resources Budget; 

• Internal Quality Assurance (monitoring); 

• Strengths/weaknesses. 

Thereafter, the five contributions were screened on the current and planned practice in 
an agency related to the ten benchmark topics. What is the agency actually doing at this 
time and what are their plans for the coming period? The results were presented in an 
overview and sent to the five agencies with a request for comments. After processing 
their response the overview was completed. Ideally the next step would have been to 
discuss the results in a meeting of the five agencies more thoroughly. But, as mentioned 
earlier, this was not feasible. However, the meeting during the seminar, where the 
preliminary results were presented, was considered to be an acceptable and pragmatic 
alternative. Fruitful discussions about staff development were held and brought this case 
further. 

The results of the comparison from the five agencies to the 10 benchmark points are 
presented here. 

A short characteristic of the activities of the group of five agencies is useful. 

All agencies are involved in evaluation and/or accreditation. Three agencies have 
programme assessment or accreditation as main activity (ACQUIN, ANECA and EVA), 
and two agencies apply institutional audits (QAA and NVAO). One agency is in the 
position to combine institutional audit and programme assessment (NVAO). 

Furthermore, two agencies are involved in specific assessment activities as 
evaluations of teaching and learning and evaluation of staff (ACQUIN and ANECA). To 
get an impression of the size of the agencies, the number of staff gives a rough indication. 
QAA and EVA are big agencies with more than 100 employed staff. EVA is not only 
involved in higher education; its higher education department is therefore smaller (32 
staff members). ANECA counts 86 staff members. NVAO is quite big as well (about 50 
staff members). Finally, ACQUIN is smaller with 22 staff members. 

These similarities and differences make a benchmark on Staff development not always 
easy, but the five agencies are comparable enough to learn from each other on specific 
aspects of staff development. 

REQUIRED AND CURRENT COMPETENCES OF STAFF 

In this benchmark activity, we focused on the professional staff of agencies; that 
means staff acting on policy level in the agency and (mostly) not part of the evaluation 
committee or an evaluation panel. Professional roles of these staff members are often 
policy advisor, process or project coordinator and/or sometimes secretary of the 
committee or evaluator of panel or committee reports. Staff members can be auditors as 
well. It turned out that, to be successful in these roles, at least a degree on Bachelor level, 
but mostly on Master level, is required in the agencies. Most staff in our five agencies 
hold a Master's level degree. (Nearly) all agencies have employees with a doctorate 
degree on the level of professional staff, but PhD employees are a minority. Work 
experience in the higher education sector is mostly required, but this differs between 
agencies. The discussion during the seminar showed that there is a need for staff 
with mixed backgrounds. It is important to have senior staff with experience from the 
institutions. But there is also need for less experienced staff at an earlier stage of their 
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career. Staff having a background solely from private consulting companies/industry 
might lack the necessary understanding of the political context. 

The needs for improvement of staff skills differ between the agencies and vary from 
improving skills on leadership and management to increasing efficiency in performance 
and improving languages, communication and teamwork skills. The importance of 
sharing experiences and cases is mentioned nearly by all. We might call it the need 
for inter-vision or learning from colleagues. In any case, the need for improvement 
is determined on an individual basis in all agencies during the annual performance 
interviews. 

STAFF ASSESSMENT 

Staff Assessment is nearly always done on an annual basis in all agencies. The individual 
performance of staff is assessed and targets for personal development are indicated. An 
interim update after six months occurs in QAA and EVA. A personal development plan 
is applied in only one agency; another agency (NVAO) has planned it for the near future. 
In two agencies (EVA and QAA) the results of the performance interviews give input 
to the training and development strategy of the agency. A preliminary conclusion could 
be that staff assessment is in place, but the relation between the results of individual 
performance, a personal development plan and the development strategy of the 
organisation can be strengthened in most agencies. The bigger agencies seem to have an 
advantage here because of their size and structure (e.g. with a separate Human Resource 
Department or Manager). 

STAFF DEVELOPMENT 

Staff development activities in the five agencies show a wide variety. In bigger agencies, 
the approach is rather structured in a training (or career) and development plan. Existing 
examples in the five agencies are: 

• information talks (e.g. inviting guest speakers from outside or inside); 

• advanced user scheme (colleagues support for new or inexperienced colleagues); 

• a leadership programme when it fits the needs of the agency; 

• courses in evaluation theory and methodology; 

• courses in quality assurance; 

• a programme in project management; 

• courses in communication and languages; 

• courses in report writing and participating in (inter)national conferences are usual 
in all agencies. 

Sometimes internships and secondments are used (e.g. a temporary place in the higher 
education sector). Inter-vision (learning from cases and practice) is explicitly applied in 
two agencies. In all agencies, staff development activities are always based on individual 
needs. 

In conclusion, a variety of activities exist that fit to the direct needs in the organisation. 
In some agencies, staff gets facilities to study for a higher degree (e.g. a doctorate 
degree) which means an investment for the long run. 

SELECTION AND GUIDANCE OF NEW STAFF AND TRAINING ON THE 
JOB 

New staff is mostly guided by a senior staff member (a mentor) during the first six 
months. An introduction programme in place is usual in all agencies. Training on the job 
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is practiced by participating in existing activities. Exchange of knowledge and experience 
between staff is a current practice in the five agencies, e.g. through inter-vision meetings, 
cross-reading reports (four eyes principle), to assure true outcomes and results of 
assessments and accreditation. Internal job rotation is often mentioned to train on the 
job; however, job rotation with external partners seldom occurs. 

REWARDS/INCENTIVES AND HUMAN RESOURCES BUDGET 

Agencies rarely give additional financial rewards. The reward is most of the time 
education and training oriented or related to other development opportunities. The key 
figures gathered to the human resources budget are not easy to compare as the agencies 
use different ways to present them. It seems that between 350 and 600 euro is spent 
per employee every year in agencies of a substantial size. This could relate to about 3 to 
4 percent of the total annual budget of the agency. 

INTERNAL QUALITY ASSURANCE / MONITORING STAFF DEVELOPMENT 

The monitoring of staff development also differs between agencies. Some agencies 
have systems where systematic evaluation of their work results in systematic evaluation 
of staff performance as well. Some agencies (QAA, ACQUIN) use key performance 
indicators, for instance quarterly (QAA). Other agencies use also feedback from staff 
satisfaction surveys (NVAO and EVA). Staff development can follow a more qualitative 
approach as well. For example, EVA monitors staff development through appraisal 
interviews twice a year with a systematic follow up without using key performance 
indicators. 

STRENGTHS/WEAKNESSES 

When it comes to strengths and areas to improve, all agencies have different priorities. 
For instance, QAA is strong in 'in house' expertise in training and development and 
has advanced systems, but these activities can still be further developed. EVA is 
strong in qualitative processes, but has to cover many different types of evaluations 
besides quality assurance work, which gives staff variation of tasks. In this situation, it 
is challenging to manage staff competences and development. ACQUIN, as a smaller 
organisation, focuses on a need for more systematic staff development but is strong 
in its ability to make room for individual needs. ANECA offers good job conditions 
and has already well defined procedures regarding professional development with 
training and self improvement and training based on individual needs as languages 
and ICT skills. Still, ANECA wishes to develop more certain areas. And finally, NVAO 
intends to focus more on the use of personal development plans and the assessment of 
competences of current staff. 

CONCLUSION 

The benchmark on staff development can be assessed as a valuable first step. The 
exercise carried out for the IQA seminar was a good exercise from which the five 
agencies have already learned a lot. At the same time, they all agree that 'the job is not 
completed yet', but has just started. A good comparison as an essential part of a good 
benchmark requires that all parties involved are well prepared and meet face-to-face. 
These conditions are needed to make further progress. Face-to-face meetings will 
be difficult to organise, but it is important to look for more support and facilities. The 
development of web conferencing or 'webinars' could assist in crossing boundaries. 
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Nevertheless, this first benchmark activity demonstrates that staff development is a 
significant issue into which agencies put substantial effort. As professional staff is the 
'core capital' of a quality assurance agency, this might be self evident. Benchmarking on 
staff development is an opportunity to gather new ideas and inspiration. 
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CHAPTER 4: 

BENCHMARKING OF 
PERFORMANCE INDICATORS 

Matti Kajaste, Finnish Higher Education Evaluation Council (FINHEEC) 

Luis Carlos Velon Sixto, Agency for Quality Assurance in the Galician University System 
(ACSUG) 

Harald Scheuthle, Stiftung Evaluationsagentur Baden-Wuerttemberg (evalag) 

Nuria Comet, Catalan University Quality Assurance Agency (AQU Catalunya) 

Sergio Machado dos Santos, Agency for Evaluation and Accreditation of Higher Education 
(A3ES) 


INTRODUCTION 

The annual seminar of the ENQA Internal Quality Assurance Group (IQA), held in June 
2011, concentrated in the use of benchmarking as a tool for learning and developing the 
quality assurance agencies' respective internal QA systems. The IQA Steering Group set 
up three different topics for benchmarking. The topic of performance indicators turned 
out to be so popular that two different subgroups, each composed of three agencies, 
were formed. 

SUBGROUPONE 

The first subgroup consisted in ACSUG (Spain), evalag (Germany) and FINHEEC 
(Finland). The three agencies had a meeting in Mannheim, hosted by evalag on 26th 
April 2011. The partners first prepared short presentations on their activities, internal 
quality assurance systems and the ways in which performance indicators are used. 

Different approaches 

ACSUG 

The Agency for Quality Assurance in the Galician University System was founded in 
2001 as a joint project of the Galician Government and the three Galician universities. 
The key mission of the ACSUG is to contribute towards the improvement of the Galician 
University System by issuing reports, conducting assessments, providing certification 
and accreditation for Galician University activities, particularly those related to teaching, 
research, the transfer of knowledge, and management. The ACSUG also acts as an 
instrument for the permanent collection and channeling of information between the 
Galician universities, other institutions and stakeholders, enabling the Galician University 
System to remain up to date and in touch with changing social demands. 

ACSUG, as a quality assurance agency, is also subject to quality criteria and has 
established, and externally certified, a Quality Management and Environment System, 
according to the ISO 9001:2008 and ISO 14001:2004 regulations and the EM AS 
Regulation. This system is based on continuous improvement to obtain efficient and 
effective performance of its activities. ACSUG has identified, into its management 
system, the different procedures needed for the good functioning of the agency and it 
has established different indicators to measure the quality of its activities. 
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evalag 

evalag was founded in 2001 by the state of Baden-Wiirttemberg as a Foundation under 
public law. Since 2009, evalag is recognised as an accreditation agency in Germany. Its 
main objective is to support higher education institutions in developing their internal 
quality. Therefore, evalag offers a broad range of services and external quality assurance 
procedures such as consulting services, evaluations, quality audits and accreditations. In 
supporting higher education institutions, evalag follows an enhancement-led approach 
that focuses on developing the internal quality management processes in institutions. 

In its own internal quality assurance, evalag strongly relies on qualitative methods, 
which are supplemented by quantitative indicators. The main improvement tool is the 
internal reflection of the quality assurance procedures carried out by evalag. On a regular 
basis (once or twice a year) all employees meet to discuss along nine pre-set topics the 
procedures and experience gained, with a view to enhancement. Additionally, evalag 
gathers online feedback questionnaires from experts and higher education institutions 
after each procedure. The feedback received is used for the internal reflection as well. 

FINHEEC 

The Finnish Higher Education Evaluation Council was founded in 1996. It currently 
conducts three different types of evaluations: institutional audits of quality assurance 
systems of mainly Finnish higher education institutions, thematic evaluations on different 
topics related to higher education and evaluations of centres of excellence in both 
university and university of applied sciences sectors. FINHEEC is an independent expert 
body assisting universities, universities of applied sciences and the Ministry of Education 
and Culture in matters related to evaluation, thus contributing to the improvement of 
higher education. FINHEEC adopts a profoundly enhancement-led approach with a 
strong emphasis on supporting the higher education institutions to improve the quality of 
provision. FINHEEC is exclusively funded by the government. 

FINHEEC gathers feedback systematically after all evaluations, both from institutions 
and review panels. The feedback is used in the annual development seminars of the 
FINHEEC Secretariat and Evaluation Council. The changes made to operations and 
processes are then communicated to the feedback providers. From the feedback a 
number of indicators are drawn. These relate to the quality of the output, normally 
an evaluation report, of the evaluation process and of the evaluation team selected. 

In addition, FINHEEC utilises other indicators on the activities of the Secretariat (for 
example, lectures given, seminars held and participated in). The feedback is reported to 
the Evaluation Council annually. 

Similarities 

The striking similarities between the three benchmarking partners made learning 
from each agency immediate. First of all, all three agencies conduct audits of higher 
education institutions, which means that they were all aware of the possibilities and 
limitations related to such evaluation methods, evalag's approach is perhaps even more 
enhancement-led than FINHEEC's since the former tailor-makes the evaluation method 
for each individual institution. FINHEEC, on the other hand, uses the quality audit model 
as the national quality assurance method and therefore all audits are uniform with the 
exactly same criteria. 

Secondly, all three agencies utilise online questionnaires for gathering feedback 
and building 'customer satisfaction’-type surveys from which the most important 
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performance indicators are drawn. The web-based facilities make it more time efficient 
to gather, use and disseminate the feedback information and to calculate the indicators. 

Thirdly, all benchmarking partners report the annual performance indicator readings 
to their respective councils annually. Together with the evaluation council's expertise and 
experience, more profound changes to operations and processes can be made. 

Finally, the objectives which ACSUG, evalag and FINHEEC have set on their internal 
quality assurance are strikingly similar. The IQA work is mostly done to improve the 
internal operations of the agency and to assure the quality of evaluations. All agree 
on the most important issues that internal quality should focus on and these are also 
reflected by the chosen performance indicators. 

Differences 

Most of the differences in the use of performance indicators arise from the differences 
in the evaluations themselves and the way they are organised. Due to the relative 
uniqueness of evalag's evaluations, its processes cannot be easily compared and 
therefore its working performance indicators are difficult to adopt by other agencies. 

Also, evalag has so far conducted only a fairly limited amount (two at the time of 
the benchmarking exercise) of institutional audits and repeatable patterns have yet 
to emerge. FINHEEC has more experience in conducting institutional audits and its 
evaluation processes are aimed at being fairly uniform. Therefore performance indicators 
are easier to apply. ACSUG, by comparison, uses an Internal Management System 
based on the standards ISO-9001 (quality), ISO 14001 and EMAS (environment) with a 
fairly robust set of indicators. ACSUG supports a much wider range of indicators than 
FINHEEC, both in terms of indicators monitoring the quality of the output and the quality 
of the evaluation process itself. ACSUG has also appointed a special technician for 
coordinating the internal quality work and performance indicators, while other agencies 
have dedicated fewer resources to these issues. 

Observations 

The benchmarking partners prepared a presentation for the annual IQA seminar held in 
Helsinki, Finland in June 2011. The presentation highlighted a number of issues that the 
partners felt might be of benefit to the participants of the seminar. 

Firstly, FINHEEC shared a problem common to many or most quality assurance 
agencies. Most of the feedback questionnaires and other forms of dialogue with higher 
education institutions actually reach only the very top leadership of the institution, 
which means that only the leadership voice is heard, at least directly, by the agency. 

This may become a dangerous illusion, as the leadership cannot entirely represent the 
opinion of the whole staff. The benchmarking partners suggested the possibility to 
conduct a number of feedback surveys, with separate questionnaires designed to collect 
information from non leadership staff members or students, evalag started sending 
feedback questionnaires to all site visit participants in order to gather a broader feedback 
on its procedure and on the reception of the recommendations in the institution. 

Secondly, ACSUG shared an indicator that could be useful to other agencies as well. 
ACSUG follows the involvement of staff - in internal quality assurance - who are in 
charge of collecting data for either individual indicators or ownership of processes. This 
was seen by the benchmarking partners as a useful way to spread responsibility and 
strengthen the commitment of the whole organisation on internal quality assurance. 

Thirdly, ACSUG also follows, as a part of its wide range of indicators, the percentage of 
planned trainings in which staff members have actually participated. In addition, ACSUG 
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gathers feedback from staff on the usefulness and quality of the trainings to improve 
them. 

Final thoughts 

The three agencies found the benchmarking exercise extremely useful. The partners 
learned about very different ways to build an internal quality assurance system and 
different approaches to performance indicators. A number of tacit-knowledge pieces of 
information were also shared on building and running performance indicator schemes. 
This method of fairly personal benchmarking provided the parties with an opportunity to 
freely discuss the possibilities, ideas and concerns related to indicators and IQA systems. 
Also, the partners strongly felt that the face-to-face meeting greatly facilitated the free 
exchange of ideas and experiences. The official, printed information is fairly easy to 
share by just sending out documents, but an important role in running an internal quality 
assurance system is played by tacit knowledge, which cannot be baked into a pdf. 

SUBGROUP TWO 

In subgroup two, the partners were AQU Catalunya (Spain) and A3ES (Portugal). 

First of all, the two partners compared their agency and the way their internal quality 
system is managed. They prepared a short presentation on their activities, internal 
quality assurance systems and the way performance indicators are used. 

AQU Catalunya 

AQU Catalunya is the primary body responsible for quality promotion and assurance 
in the Catalan higher education system. It is a public agency subject to private law, 
operating under the authority of the government department which has jurisdiction over 
universities. It is a separate legal entity with full legal capacity to act and has its own 
privileges and liabilities. 

Set up as a consortium between the Catalan Government (Generalitat de Catalunya) 
and the universities in 1996, the Agency was the first quality assurance agency 
established in Spain. With the passing of the Catalan Universities Act (2003), the 
Agency was renamed (Agenda per a la Qualitat del Sistema Universitari de Catalunya). 
The purpose of AQU Catalunya is to carry out the evaluation, accreditation and 
certification of quality in the universities and higher education institutions in Catalonia. 

Since 1996 the implementation of the quality management system at AQU Catalunya 
reflects the willingness to implement a mechanism that allows the organisation to 
be managed in a systematic and visible way both for the management team and for 
all staff. The system gathers the results of all processes thorough quantitative and 
qualitative indicators, as well as the feedback from all the stakeholders involved in the 
process (experts, institutions, AQU staff). At the end of each project, AQU compiles a 
final document including the feedback, self-evaluation and indicators. Every year, AQU 
conducts the annual review of the internal quality system, resulting in a report. AQU is 
ISO 9001 certified, which implies two annual mandatory reviews (internal and external) 
of the internal quality system. 

A3ES 

A3ES is a private law foundation created by the State, independent in its decisions, with 
responsibility for the assessment and accreditation of Portuguese higher education 
institutions and their study programmes, and the promotion of the integration of Portugal 
in the European system of quality assurance in higher education. Currently, the agency 
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is concentrating its work on compulsory ex-ante and ex-post accreditation of degree 
programmes and is launching a voluntary process of institutional audit. 

Internal quality assurance of A3ES relies both on formal and informal mechanisms. 
Formal mechanisms include feedback questionnaires from institutions, experts and 
process managers, as well as an annual visit and review by a fully international Scientific 
Committee of well recognised experts. Informal mechanisms, more adaptable to the 
small size of the agency, consist of the frequent contacts of the Board members with 
all the collaborators and with the institutions. Research projects on quality assurance, 
carried out by the Development and Analysis Office, contribute to the agency's 
continuous enhancement. 

Objective 

The objective of the group was to share information about indicators and assess if they 
were usable to compare the performance of the Agencies. 

Development 

The actions taken were: 

1st activity: define the classification of indicators 

2nd activity: define characteristics of each indicator 

3rd activity: agree which are the common indicators in each group 

4th activity: take some conclusions. 

As regards the classification of indicators, the group decided to make the classification 
according to the aspects to be measured: 

• Level of Activity: quantitative measure of the different types of activities performed 
by the Agency concerning information, documentation and assessments; 

• Internal and external capacity building: indicators on the qualification and training 
of internal staff and staff involved in QA activities within HEIs; 

• Impact: quantitative measure of the effect of the Agency's activity on HEIs and 
society; 

• Resources: indicators related to the Agency's financial and human resources; 

• Satisfaction: qualitative indicators about satisfaction of Agency's internal and 
external activities. 

The characteristics of each indicator were defined: 

1. Name and description 

2. Method to collect the information 

- Manual 

- Data Base/ Electronic Platform 

3. Periodicity 

- Annual 

- At any time 

4. Categorisation 

- By Institution's typology (university, polytechnic, public, private, etc.) 

- By assessment 

5. Public Information 

- Published on the web: public report 

- Internal report: agency report 
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Although only two Agencies took part in the exercise, some common indicators were 
nevertheless found, and are summarised in the following table: 


TYPE 

INDICATOR 

COMMENTS 

Level of activity 

Nb. of institutional assessments 

Audit assessments 

Nb. of programme assessments 

Assessments (leading or not an 
accreditation decision ) 

Nb. of international assessments 

Assessments done in other countries. 

Nb. of HEIs involved 

Nb. of HEIs that submitted programmes 
for accreditation each year 

Results 

Nb. of positive results 

Nb. of proposals with a positive decision 

Nb. of negative results 

Nb. of proposals with a negative 
accreditation decision 

Nb. of conditional decisions 

Nb. of proposals with a conditional 
accreditation decision 

Level Activity - 
Information 

Nb. of reports 

Nb. of reports from the review panels, 
concerning any activity 

Nb. of publications 

Nb. of papers, reports, books published 
by the Agency (paper or digital versions) 

Nb. of news on the website 


Level Activity - 
Information 

Nb. of studies 

Nb. of thematic or cross-sector studies 
undertaken by the R&D Unit 

Internal and external 
capacity building 

Nb. of hours of training 
(for internal staff) 


Nb. of training actions done for internal 
staff 


Nb. of training actions done for staff of 
HEIs 


Nb. of conferences or seminars (in HEIs 
or others) with participation of the 
Agency 


Resources 

Total staff 


Nb. of technical staff 


Nb. of external experts participating in 
review panels (national / international) 


Total budget 


Annual costs with external experts 
(payment and logistic cost) 


Annual costs with staff 


Impact 

Nb. of visits to the website 


Satisfaction 

Surveys 

Nb. of comments in each report 

Qualitative indicators, not numerical. 
Difficult to compare 
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CONCLUSIONS 

Following the comparison exercise of the two Agencies, these conclusions may be drawn: 

Is it possible to compare Agencies? 

• The agencies work in very different contexts, using different processes although 
some have comparable procedures; 

• The indicators used for this benchmarking exercise may seem, at first sight, to be of 
little value and have little meaning if considered alone. They are context sensitive; 

• More data from different agencies would be needed to choose the best and most 
representative indicators. 

Could indicators be a tool to compare the performance of the agencies? 

• It is difficult and lengthy to compare agencies with this type of indicators; 

• They are a good internal tools to monitor and improve the effectiveness of the 
quality management system; 

• It is useful to know which indicators other agencies use (qualitative comparison). 

FINAL THOUGHTS 

Some areas to work on in the future were identified: 

• The group considered that it could be more important to proceed with the exchange 
of practices on the use of different procedures; 

• Nevertheless, when exchanging and comparing practices, agencies should also 
look at the way(s) in which they measure the impact of such practices (internal: 
resources; and external: results) 

• It would be important to develop meaningful indicators to assess the impact of the 
agency's work on HEIs and on the HE system as a whole. 
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CONCLUSIONS 

The 2011 seminar "Learning from each other - using benchmarking to develop IQ A" revealed 
the constant interest of benchmarking as a tool for quality improvement in quality 
assurance agencies. 

The benchmarking exercise and the resulting articles of this report provided 
interesting reflections on the following focus areas: 

FOLLOW-UP OF FEEDBACK 

This session underlined the importance of gathering correct information from 
stakeholders, not only using surveys but also through informal meetings or discussion 
groups. It pointed out also the difficulty of pinpointing the strategic "stakeholders" when 
gathering information and the problem of implementing improvements in the process 
when it is imposed by legislation or by government authorities. 

STAFF COMPETENCE/DEVELOPMENT 

This session arrived to the major conclusion that staff assessment is considered as a tool 
to improve competences but needs to be further developed. A wide variety of activities 
for staff development exist within the agencies, ranging from a structured development 
plan to individual courses. Agencies rarely prefer using financial rewards and incentives 
but rather use alternative schemes such as education and training. 

PERFORMANCE INDICATORS 

First of all, the use of performance indicators reported to the board/council of the agency 
helps to improve the operations and processes carried out in the agency. At the same 
time, the different use of performance indicators in each agency makes it difficult to 
compare agencies. It is however clear that, involving the staff in collecting data is a way 
to make the whole agency feel responsible for IQA: it is important to develop meaningful 
indicators to assess the impact of agency's work. 

All the groups felt that, although new technology helps to share information, face- 
to-face meetings were most adequate to freely exchange ideas and experiences and, 
consequently, obtain better benchmarking results. The seminar reached its objective, 
that is, to present findings of an effective benchmarking exercise between agencies 
having a similar profile and common interests. 
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ANNEX 


INTERNAL QUALITY ASSURANCE GROUP SEMINAR 
16 AND 17 JUNE 2011 
HELSINKI, FINLAND 

HOSTED BY FINNISH HIGHER EDUCATION EVALUATION COUNCIL (FINHEEC) 

Venue 

Ministry of Education and Culture 
Auditorium Jukola 
Meritullinkatu 1 
Helsinki, Finland 


PROGRAMME 

DAY 1: 16 JUNE 2011 

Chair: Matti Kajaste, FINHEEC 

11:30 Registration 

12:00 Lunch at restaurant Sipuli, Kanavaranta 7 

13:00 Opening session 

Tine Holm, EVA, ENQA Board 
Marja-Liisa Saarilammi, FINHEEC 
Matti Kajaste, FINHEEC 

13:30 Keynote speech on the theme of Benchmarking 

Nadine Burquel, Secretary-General of the European centre of Strategic 
Management of Universities (ESMU) 

15:00 Break 

15:30 Presentation of the benchmark topic "Performance Indicators" 

Matti Kajaste, FINHEEC and Nuria Comet Serial, AQU Catalunya 

16:00 Discussion in small groups about the experiences and results on this theme 
in all participating agencies 

17:00 End of day one and introduction to the social programme 
Matti Kajaste, FINHEEC 

19:30 M/S Panda embarks from Kauppatori harbour for a cruise in Helsinki 
archipelago 

20:30 Dinner at Nylandska Jaktklubben 
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DAY 2: 17 JUNE 2011 


Chair: Matti Kajaste, FINHEEC 

8:30 Election of Steering group 
Henri Ponds, NVAO 

9:00 Presentation of the benchmark topic "Staff development" 

Henri Ponds, NVAO 

9:30 Discussion in small groups about the experiences and results on this theme 
in all participating agencies 

10:30 Break 

11:00 Presentation of the benchmark topic "Follow-up on feedback" 

Sandra Marcos, ACSUCYL 

11:30 Discussion in small groups about the experiences and results on this theme 
in all participating agencies 

12:30 Lunch at restaurant Sipuli, Kanavaranta 7 

13:30 Plenary feedback of the three small table sessions on the three benchmark 
topics 

Maiki Udam, EKKA and Douglas Blackstock, QAA 

14:30 Feedback session from the seminar and ideas for next year 
Matti Kajaste, FINHEEC 

15:00 Conclusions 

Matti Kajaste, FINHEEC 
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